Image processing device, method of processing image, image processing program, and imaging device

ABSTRACT

An information processing system configured to perform predetermined processing based on out-of-focus information corresponding to a first area of an image, the out-of-focus information corresponding to a difference in focus between the first area of the image and a second area of the image that is different from the first area of the image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation Application of application Ser. No.14/894,435, filed Nov. 27, 2015 which is a 371 Nationalization ofPCT/JP2014/070143, filed Jul. 22, 2014 and claims the benefit ofJapanese Priority Patent Application JP 2013-152937 filed on Jul. 23,2013, the entire contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present technology relates to an image processing device, a methodof processing an image, an image processing program, and an imagingdevice.

BACKGROUND ART

In recent years, imaging devices, such as digital cameras, have beenwidely spread, and there are some imaging devices provided with afunction to automatically and optimally adjust focus and exposureaccording to an image capturing scene. However, special stage effects,such as adjustment of a composition and shading off of a background atthe time of capturing an image, are still largely depending on skills ofusers. Especially, for beginners who do not have knowledge of cameras,it is difficult to capture an image with the special staging effects.Therefore, in recent years, many technologies to shade off thebackground, and the like have been proposed (PTL 1).

CITATION LIST Patent Literature

[PTL 1]

-   JP 2008-233470A

SUMMARY

Many of related technologies detect an object, and set a diaphragmaccording to information of the object. To be specific, the technologiesdetect a face of a person, and set a diaphragm according to informationof the size, position, direction or the like of the face. Such adiaphragm determination technique is automatically controlled, and thusthere is a possibility that an image is captured against intension ofthe user.

The present technology has been made in view of such a problem, andthere is a need for providing an image processing device, a method ofprocessing an image, an image processing program, and an imaging devicethat enable easy acquisition of a blurred image by performing processingbased on information related to blur in the image.

Solution to Problem

To solve the above problem, a first technology is an informationprocessing system configured to perform predetermined processing basedon out-of-focus information corresponding to a first area of an image,the out-of-focus information corresponding to a difference in focusbetween the first area of the image and a second area of the image thatis different from the first area of the image.

Advantageous Effects of Invention

According to an embodiment of the present technology, processing isperformed based on information related to blur in an image, whereby theuser can easily acquire a blurred image.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of an imageprocessing device according to an embodiment of a present technology.

FIG. 2 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a first embodiment.

FIG. 3 is a diagram for describing a method of calculating a blurprediction amount.

FIGS. 4A and 4B are diagrams illustrating display forms in the firstembodiment.

FIG. 5 is a diagram illustrating a display form in the first embodiment.

FIG. 6 is a diagram illustrating a display form in the first embodiment.

FIG. 7 is a flowchart illustrating a flow of processing in the firstembodiment.

FIG. 8 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a second embodiment.

FIGS. 9A to 9C are diagrams illustrating display forms in the secondembodiment.

FIG. 10 is a flowchart illustrating a flow of processing in the secondembodiment.

FIG. 11 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a third embodiment.

FIGS. 12A and 12B are diagrams illustrating display forms in the thirdembodiment.

FIGS. 13A and 13B are diagrams illustrating display forms in the thirdembodiment.

FIG. 14 is a flowchart illustrating a flow of processing in the thirdembodiment.

FIG. 15 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a fourth embodiment.

FIGS. 16A to 16C are diagrams illustrating display forms in the fourthembodiment.

FIG. 17 is a flowchart illustrating a flow of processing in the fourthembodiment.

FIG. 18 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a fifth embodiment.

FIGS. 19A and 19B are diagrams illustrating display forms in the fifthembodiment.

FIG. 20 is a flowchart illustrating a flow of processing in the fifthembodiment.

FIG. 21 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a sixth embodiment.

FIGS. 22A and 22B are diagrams illustrating display forms in the sixthembodiment.

FIG. 23 is a flowchart illustrating a flow of processing in the sixthembodiment.

FIG. 24 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a seventh embodiment.

FIGS. 25A and 25B are diagrams illustrating display forms in the seventhembodiment.

FIG. 26 is a flowchart illustrating a flow of processing in the seventhembodiment.

FIG. 27 a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto an eighth embodiment.

FIGS. 28A and 28B are diagrams describing segmentation of an image inthe eighth embodiment, and FIG. 28C is a diagram illustrating ahistogram used for composition determination.

FIGS. 29A to 29C are diagrams describing a second example of imagesegmentation in the eighth embodiment.

FIG. 30 is a flowchart illustrating a flow of processing in the eighthembodiment.

FIG. 31 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a ninth embodiment.

FIGS. 32A to 32D are diagrams illustrating images acquired by diaphragmbracket image capturing in the ninth embodiment.

FIGS. 33A to 33D are diagrams illustrating a second example of an imageacquired by the diaphragm bracket image capturing in the ninthembodiment.

FIG. 34 is a flowchart illustrating a flow of processing in the ninthembodiment.

FIG. 35 is a block diagram illustrating a configuration of an imagingdevice provided with a function of an image processing device accordingto a tenth embodiment.

FIGS. 36A and 36B are diagrams illustrating an example an image to whichimage processing has been applied in the tenth embodiment.

FIG. 37 is a flowchart illustrating a flow of processing in the tenthembodiment.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present technology will be describedwith reference to drawings. Note that description will be given in thefollowing order.

-   1. First Embodiment-   [1-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [1-2. Processing in an image processing device and an imaging    device]-   2. Second Embodiment-   [2-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [2-2. Processing in an image processing device and an imaging    device]-   3. Third Embodiment-   [3-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [3-2. Processing in an image processing device and an imaging    device]-   4. Fourth Embodiment-   [4-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [4-2. Processing in an image processing device and an imaging    device]-   5. Fifth Embodiment-   [5-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [5-2. Processing in an image processing device and an imaging    device]-   6. Sixth Embodiment-   [6-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [6-2. Processing in an image processing device and an imaging    device]-   7. Seventh Embodiment-   [7-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [7-2. Processing in an image processing device and an imaging    device]-   8. Eighth Embodiment-   [8-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [8-2. Processing in an image processing device and an imaging    device]-   9. Ninth Embodiment-   [9-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [9-2. Processing in an image processing device and an imaging    device]-   10. Tenth Embodiment-   [10-1. Configurations of an image processing device and an imaging    device including the image processing device]-   [10-2. Processing in an image processing device and an imaging    device]-   <11. Modifications>

1. First Embodiment

[1-1. Configurations of an Image Processing Device and an Imaging DeviceProvided with the Image Processing Device]

First, configurations of an image processing device 1 and an imagingdevice 1100 provided with a function of the image processing device 1according to a first embodiment will be described. FIG. 1 is a blockdiagram illustrating the image processing device 1.

The image processing device 1 is configured from an attention areadetermination unit 11, a blur information acquisition unit 12, and ablur related processing unit 13. The image processing device 1 isoperated in an imaging device, for example.

FIG. 2 is a block diagram illustrating a configuration of the imagingdevice 1100 provided with a function of the image processing device 1.The imaging device 1100 is configured from an optical imaging system1001, a lens control unit 1002, an imaging element 1003, a preprocessingcircuit 1004, a camera processing circuit 1005, an image memory 1006, adisplay unit 1007, an input unit 1008, a storage unit 1009, and acontrol unit 1010. Further, the control unit 1010 functions as an objectdetection unit 1011, a focus determination unit 1012, and an attentionarea determination unit 11, a blur information acquisition unit 12, anda blur related processing unit 13, which configure the image processingdevice 1. In the first embodiment, the blur related processing unit 13functions as a graph creation unit 14 and a display control unit 15.

The optical imaging system 1001 is configured from an image capturinglens for collecting light from an object to the imaging element 1003, adrive mechanism for moving the image capturing lens and performingfocusing and zooming, a shutter mechanism, an iris mechanism, and thelike. These lens and mechanisms are driven by control of the lenscontrol unit 1002. An optical image of the object obtained through theoptical imaging system 1001 is formed on the imaging element 1003.

The lens control unit 1002 is a microcomputer in a lens, for example,and controls operations of the drive mechanism, the shutter mechanism,and the iris mechanism of the optical imaging system 1001, and the like,according to control from the control unit 1010 and the image adjustmentunit 12. This enables adjustment of an exposure time (shutter speed), adiaphragm value (F-number), and the like.

The imaging element 1003 photoelectrically converts incident light froman object to convert it into a charge amount, and outputs the chargeamount as an analog imaging signal. The analog imaging signal outputfrom the imaging element 1003 is output to the preprocessing circuit1004. As the imaging element 1003, a charge coupled device (CCD), acomplementary metal oxide semiconductor, or the like is used.

The preprocessing circuit 1004 samples and holds the imaging signaloutput from the imaging element 1003 by correlated double sampling (CDS)processing to favorably maintain a signal/noise (S/N) ratio. Further,the preprocessing circuit 1004 controls a gain by auto gain control(AGC) processing, performs analog/digital (A/D) conversion, and outputsa digital image signal. These pieces of processing are performedaccording to control from the control unit 1010.

The camera processing circuit 1005 applies, to an image signal from thepreprocessing circuit 1004, signal processing, such as white balanceadjustment processing, color correction processing, gamma correctionprocessing, Y/C conversion processing, and auto exposure (AE)processing.

The image memory 1006 is a volatile memory, for example, a buffer memoryconfigured from a dynamic random access memory (DRAM). The image memory1006 temporarily stores image data to which predetermined processing hasbeen applied by the preprocessing circuit 1004 and the camera processingcircuit 1005.

The display unit 1007 is a display device configured from a liquidcrystal display (LCD), a plasma display panel (PDP), an organic electroluminescence (EL) panel, or the like, for example. In the display unit1007, a through image during imaging, a captured image recorded in thestorage unit 1009, and the like are displayed.

The input unit 1008 is made of a power button for switching powerON/OFF, a release button for instructing start of recording of animaging image, an operator for zoom adjustment, a touch screenintegrally configured with the display unit 1007, and the like. When aninput is given to the input unit 1008, a control signal according to theinput is generated and output to the control unit 1010. The control unit1010 then performs arithmetic processing and control corresponding tothe control signal.

The storage unit 1009 is a mass storage medium, such as a hard disk, amemory stick (a registered trademark of Sony Corporation), or an SDmemory card. An image is stored in a compressed state based on astandard, such as Joint Photographic Experts Group (JPEG), for example.Further, exchangeable image file format (EXIF) data includinginformation related to the stored image and additional information suchas a date and time of imaging is also stored in association with theimage.

The control unit 1010 is configured from a central processing unit(CPU), a random access memory (RAM), and a read only memory (ROM), andthe like. In the ROM, programs read and operated by the CPU are stored.The RAM is used as a work memory of the CPU. The CPU executes varioustypes of processing according to the programs stored in the ROM, andcontrols the entire imaging device 1100 by issuing commands.

Further, the control unit 1010 functions as the object detection unit1011, the focus determination unit 1012, and the attention areadetermination unit 11, the blur information acquisition unit 12, and theblur related processing unit 13, which configure the image processingdevice 1, by executing predetermined programs. However, theseconfigurations are not only realized by the programs, but may also berealized by a combination of a dedicated device by hardware that has theabove functions, a circuit, and the like.

Note that an object to be processed of the image processing device 1 inthe first embodiment is a frame image that configures a through image.

The object detection unit 1011 detects an object from an image to beprocessed. An example of the object includes a face of a person, forexample. Note that a thing detected by the object detection unit 1011does not necessarily need to be a face of a person, and may be ananimal, a building, or the like, as long as it can be detected. As adetection method, template matching based on a shape of an object,template matching based on luminance distribution of a face, a methodbased on a characteristic amount of a flesh-colored portion or a face ofa human included in an image, or the like, can be used. Further,detection accuracy may be enhanced by a combination of these techniques.

The focus determination unit 1012 determines whether an object detectedin an image is focused. Examples of a factor to focus an object includehalf press of a shutter by a user, full press of the shutter, autofocus, continuous auto focus (AF), manual focus, and the like. Adetermination result by the focus determination unit 1012 is supplied tothe attention area determination unit 11.

The attention area determination unit 11 determines an area to be paidattention (hereinafter, called attention area) in an image to beprocessed. The attention area is an area in which a thing that isconsidered as a principal object in the image by a user, who capturesthe image, such as a person, an animal, a plant, and a building, exists.

The attention area determination unit 11 determines the attention areabased on object information acquired by the object detection unit 1011,for example. Further, the attention area determination unit 11 maydetermine an object focused by the imaging device 1100 as the attentionarea. Further, the attention area determination unit may determine theattention area based on an input that specifies an area on a throughimage from the user (an input to a touch panel, various buttons, or thelike included in the imaging device 1100).

Further, the attention area determination unit may determine an objectcloser to a center in an image as the attention area. Further, theattention area determination unit may determine an object having apredetermined size or more in an image as the attention area.

Information that indicates the attention area in an image determined bythe attention area determination unit 11 is supplied to the blurinformation acquisition unit 12.

The blur information acquisition unit 12 acquires blur information in anarea outside the attention area (hereinafter, called non-attention area)in the image.

The blur information includes a blur amount and a blur predictionamount. The blur amount can be acquired based on a depth of field orphase difference range-finding information in a state where theattention area is focused in the imaging device 1100. The degree of bluris larger as the value of the blur amount is larger. The blur predictionamount indicates to what extent the non-attention area is blurred on theassumption that the attention area is focused. Here, the blur predictionamount as the blur information will be described with reference to FIG.3. The blur prediction amount can be expressed by the followingexpression (1):

Blur prediction amount=(A−B)/d . . . (1) where an image forming positionof the attention area is A, an image forming position of thenon-attention area is B, and a focal depth is d.

In this expression (1), only the defocus amount (A−B) of the attentionarea and the background area may just be obtained. Therefore, it is notnecessary to actually form an image on the attention area, and thedefocus amount can be obtained from various AF systems, such as acontrast detection system and a phase difference system, and an AFsensor, and the like. The degree of blur in an image may become largeras the value of the blur prediction amount is larger.

Note that the attention area and the non-attention area are notnecessarily areas having a uniform distance, and for example, theseareas may be obtained and treated such that the areas are classified andtreated for each given defocus amount, the areas may be averaged andtreated for each vicinity area on an XY plane, or the widest and thesame distance areas may be treated as representative values.

Further, the focal depth is determined based on a permissible circle ofconfusion δ, and δ differs according to an appreciation size. Therefore,a reference may be determined by a method of selecting the reference bythe user in advance (according to A3/A4/L size printing, what inchequivalent of the screen size, or the like), or a method ofautomatically selecting the reference according to the number ofrecording pixels of an image. Further, when trimming is performed, thefocal depth can be treated in an appropriate unit by multiplying acoefficient according to a trimming amount. The blur informationacquired by the blur information acquisition unit 12 is supplied to theblur related processing unit 13. The blur related processing unit 13performs various types of processing according to the blur information.In the first embodiment, the blur related processing unit 13 functionsas the graph creation unit 14 and the display control unit 15. The blurrelated processing unit 13 presents the blur information to the user bydisplaying a graph that indicates the blur information in the displayunit 1007 included in the imaging device 1100.

The graph creation unit 14 creates a graph that indicates the blurinformation based on the blur information supplied from the blurinformation acquisition unit 12. The display control unit 15 performsdisplay control of superimposing and displaying the graph generated bythe graph creation unit 14 on the image in the display unit 1007 of theimaging device 1100.

FIGS. 4A and 4B are first examples of forms of the graph generated bythe graph creation unit 14 and displayed in the display unit 1007. Inthese examples, the blur amount as the blur information is presentedwith a bar graph superimposed and displayed on the through imagedisplayed in the display unit 1007.

The blur amount and the height of the bar graph correspond to eachother, and the bar graph becomes higher when the blur amount is large,and the bar graph becomes shorter when the blur amount is small.Further, a color of the bar graph may be changed according to the heightof the bar graph so that the graph can be easily recognized by the user.Note that the presentation of the blur amount is not limited to the bargraph, and may be a pie graph.

In the example of FIG. 4A, the person is the attention area, and thebackground is the non-attention area. The non-attention area is not muchblurred. Therefore, the blur amount is small, and the bar graphdisplayed on the left on the screen is short.

Meanwhile, in the example of FIG. 4B, the person is the attention area,and the background other than the person is the non-attention area. Thenon-attention area is blurred. Therefore, the blur amount is large, andthe bar graph is high.

The blur information is visually presented in this way, whereby the usercan accurately recognize to what extent the non-attention area isblurred at present. Note that it is favorable that the blur informationacquisition unit 12 continuously acquire the blur amount during imagecapturing, and the blur related processing unit changes the graphdisplay of the blur amount in real time based on the blur mount acquiredby the blur information acquisition unit 12 on a steady basis.

FIG. 5 is a diagram illustrating a second example of graph display ofthe blur amount. In FIG. 5, a graph is displayed based on the blurprediction amount instead of the blur amount. In the bar graph of FIG.5, the height of the bar graph indicates a range of the blur amount thatcan be realized by a present attention area, a focal distance, and amaximum aperture of a lens. Further, with the mark superimposed anddisplayed in a bar graph manner, a blur prediction amount in the presentF-number is indicated.

FIG. 6 is a third embodiment of a graph generated by the graph creationunit 14, and displayed in the display unit 1007. In the thirdembodiment, the graph creation unit 14 creates a histogram of the blurprediction amount supplied form the blur information acquisition unit12. In the histogram, the horizontal axis represents the magnitude ofthe blur amount or the blur prediction amount. The blur amount or theblur prediction amount becomes larger from the left to the right.Further, the vertical axis of the histogram represents an area ratio ofa blurred area and a non-blurred area in an image. By presenting theblur information with a histogram, more detailed information can bepresented to the user.

The image processing device 1 and the imaging device 1100 having afunction of the image processing device 1 according to the firstembodiment are configured as described above.

[1-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 1and the imaging device 1100 will be described with reference to FIG. 7.FIG. 7 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied to the control unit 1010and the image processing device 1, and the focal distance and theF-number have been acquired. First in step S101, AF start processing isperformed in the imaging device 1100. The AF start processing isexecuted by various methods, such as a half press operation of ashutter, which is one of the input unit 1008, a touch operation to thetouch panel, and automatic start by detection of object change. Further,regarding a phase difference sensor, and the like, which can acquiredistance information on a steady basis, an AF function may be operatedon a steady basis without a specific input operation from the user. Whenthe AF start processing is performed, a predetermined control signal issupplied to the image processing device 1, and processing by the controlunit 1010 and the image processing device 1 is started.

Next in step S102, an object in an image is detected by the objectdetection unit 1011. Next, in step S103, an attention area and anon-attention area in the image are determined by the attention areadetermination unit 11. As described above, the attention area is aperson, for example, and the non-attention area is an area other thanthe person as the attention area, such as a background. Next, in stepS104, whether the attention area is focused is determined by the focusdetermination unit 1012. A determination result by the focusdetermination unit 1012 is supplied to the blur information acquisitionunit 12. Next, in step S105, a blur amount or a blur prediction amountin the image is acquired by the blur information acquisition unit 12.Next, in step S106, a graph indicating the blur information is generatedby the graph creation unit 14. Then, in step S106, the graph issuperimposed and displayed on a through image in the display unit 1007by the display control unit 15.

The first embodiment of the present technology is configured asdescribed above. According to the first embodiment, the degree of blurof an image to be acquired by image capturing can be presented to theuser in various forms in an easy-to-understand manner.

Accordingly, even in a case where the degree of blur of an image in athrough image is hard to understand because the size of the display unit1007 of the imaging device 1100 is small, the user can easily recognizethe degree of blur.

2. Second Embodiment

[2-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a second embodiment of the present technology will be described.In the second embodiment, an image processing device 2 is also operatedin an imaging device 1200. FIG. 8 is a block diagram illustrating aconfiguration of the imaging device 1200 provided with the imageprocessing device 2 according to the second embodiment.

The image processing device 2 according to the second embodiment isdifferent from the first embodiment in that a blur related processingunit 23 functions as an enlargement processing unit 24 and a displaycontrol unit 25. Configurations other than the above are similar tothose in the first embodiment, and thus description is omitted.

An object to be processed of the image processing device 2 in the secondembodiment is a frame image that configures a through image. The secondembodiment presents the degree of blur of an image to the user in aneasy-to-understand manner by enlarging a part of the image.

The enlargement processing unit 24 generates an image in which a rangeincluding both of an attention area and a non-attention area in theimage is enlarged. For example, when a through image is in a stateillustrated in FIG. 9A, a range in which both of the attention area andthe non-attention area are included is enlarged as illustrated in FIG.9B. An enlargement range is favorably to set such that ratios of theattention area and the non-attention area become nearly equal in theenlargement range so that a difference of the degree of blur between theattention area and the non-attention area. However, a method of settingthe enlargement range is not limited to the above example.

The display control unit 25 superimposes and displays the image enlargedby an enlargement processing unit on the through image, as illustratedin FIG. 9B. Further, a segmented enlarged image may be fully displayedin a display unit 1007, as illustrated in FIG. 9C. Further, asillustrated in FIG. 9B, the enlarged image is superimposed and displayedon the through image, and when there is an input to the enlarged image(a contact of a finger to the touch panel), the enlarged image may befully displayed in the display unit 1007, as illustrated in FIG. 9C.

Note that this enlargement display is favorably displayed for a giventime after completion of an AF operation of the imaging device 1200.Note that the blur related processing unit 23 determines whether a blurprediction amount obtained by a blur information acquisition unit 12 isa predetermined amount or more. When the blur prediction amount is thepredetermined amount or more, the blur related processing unit 23 causesthe enlargement processing unit 24 and the display control unit 25 toexecute the above enlargement display processing.

[2-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 2and the imaging device 1200 will be described with reference to FIG. 10.FIG. 10 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S201, imageenlargement processing is performed by the enlargement processing unit24. Then, in step S202, an enlarged image is superimposed and displayedon a through image by the display control unit 25.

The second embodiment of the present technology is configured asdescribed above. The second embodiment enlarges and displays a part ofan image such that both of the attention area and the non-attention areain the image are included, whereby the user can more accuratelyrecognize the degree of blur.

3. Third Embodiment

[3-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a third embodiment of the present technology will be described. Inthe third embodiment, an image processing device 3 is also operated inan imaging device 1300. FIG. 11 is a block diagram illustrating aconfiguration of the imaging device 1300 provided with a function of theimage processing device 3 according to the third embodiment.

The image processing device 3 according to the third embodiment isdifferent from the first embodiment in that a blur related processingunit 33 functions as a blur amount adjustment unit 34 and a displaycontrol unit 35. Configurations other than the above are similar tothose in the first embodiment, and thus description is omitted. Anobject to be processed of the image processing device 3 in the thirdembodiment is a frame image that configures a through image. In thethird embodiment, a blur amount adjustment function by the blur amountadjustment unit 34 is automatically started, and a blur adjustment userinterface is further automatically displayed.

The blur amount adjustment unit 34 adjusts a diaphragm of a lens andchanges a blur amount of a non-attention area by transmitting apredetermined control signal to a lens control unit 1002 of the imagingdevice 1300 according to an input to an input unit 1008 from the user.

When the blur amount of an image is adjusted by the blur amountadjustment unit 34 according to an input from the user, a bluradjustment user interface is displayed in a display unit 1007 by thedisplay control unit 35 so that the user can easily perform an input.

When a focal distance and an F-number are obtained, and an adjustablerange of a blur prediction amount is a predetermined amount or more bychanging the F-number, the blur related processing unit automaticallystarts a blur amount adjustment function by the blur amount adjustmentunit. The display control unit 35 then displays the blur amountadjustment user interface in the display unit 1007.

FIGS. 12A and 12B are diagrams illustrating states in which the userinterface of the blur amount adjustment function is displayed in thedisplay unit 1007. In the examples of FIGS. 12A and 12B, a pointer 200movable in a vertical direction by an operation of the user isdisplayed. A position of the pointer 200 is moved in conjunction withthe blur amount of an image.

The blur amount of an image is changed according to the position of thepointer 200. At an upper end of a bar 210 that indicates a moving pathof the pointer 200, an icon is displayed together with characters of“clear” to capture the non-attention area clearly without shading offthe area. Further, at a lower end of the bar 210 that is the moving pathof the pointer 200, an icon is displayed together with characters of“shade off” to blur and capture the non-attention area. When the usermoves the pointer 200 upward, the diaphragm becomes smaller, and thenon-attention area becomes clear with a less blur amount. Meanwhile,when the user moves the pointer 200 downward, the diaphragm becomeslarger, and the non-attention area is blurred with a large blur amount.

In FIG. 12A, the pointer 200 is positioned at the upward clear side, andthus the image is not much blurred. Meanwhile, in FIG. 12B, the pointer200 is positioned at the downward shading-off side, and thus the bluramount of the image is large.

Note that, as illustrated in FIG. 13A, a position that indicates arecommended blur amount is displayed, and display of an arrow, blinkingof the position, display of animation, and the like may be performed inorder to induce the user to move the pointer 200 to the position. Thisenables the user who does not understand a blur control function toeasily blur an image.

Further, as illustrated in FIG. 13B, an area other than the adjustablerange by the blur control is filled, and when the present degree ofadjustment by the blur control has reached a limit, the pointer 200 maybe blinked for a given time, and tips may be displayed. In the exampleof FIG. 13B, a tip to emphasize the degree of blur of an image isdisplayed.

Note that the form of the user interface of the blur control function isnot limited to the one illustrated in FIGS. 12A and 12B, and 13A and13B. Any interface may be employed as long as the interface can adjust ablur amount of an image.

[3-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 3and the imaging device 1300 will be described with reference to FIG. 14.FIG. 14 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S301, the blurrelated processing unit 33 causes the blur amount adjustment function bythe blur amount adjustment unit 34 to automatically start. Note that, asdescribed above, the blur amount adjustment function is favorablyautomatically started when the adjustable range of the blur predictionamount is a predetermined amount or more by changing the F-number. Next,in step S302, the display control unit 35 then displays the blur amountadjustment user interface in the display unit 1007.

The third embodiment of the present technology is configured asdescribed above. According to the third embodiment, the blur amountadjustment function is automatically started, whereby the user caneasily perform adjustment of the degree of blur. Further, a user whodoes not know the existence of the blur amount adjustment function or auser who knows the blur amount adjustment function but has a lowfrequency of use can be urged to use the blur amount adjustmentfunction.

4. Fourth Embodiment

[4-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a fourth embodiment of the present technology will be described.In the fourth embodiment, an image processing device 4 is also operatedin an imaging device 1400. FIG. 15 is a block diagram illustrating aconfiguration of the imaging device 1400 provided with a function of theimage processing device 4 according to the fourth embodiment.

The fourth embodiment is different from the first embodiment in that ablur related processing unit 43 functions as an image processing unit44, a display control unit 45, and a blur amount adjustment unit 46.Configurations other than the above are similar to those in the firstembodiment, and thus description is omitted. An object to be processedof the image processing device 4 in the fourth embodiment is a frameimage that configures a through image. The fourth embodiment presentswhat a blurred image is like to the user by displaying the image inwhich a blur amount has been adjusted in a display unit 1007.

A blur information acquisition unit 12 acquires a blur prediction amountby present setting of the imaging device 1400, and a blur predictionamount in a maximum aperture.

The image processing unit 44 generates an image in which the degree ofblur in a state where the F-number is opened is reproduced by applyingblur processing to an image. When the blur prediction amount in thepresent setting is a predetermined amount or less, and the blurprediction amount in the maximum aperture is a predetermined amount ormore, the image processing unit 44 generates an image in which thedegree of blur in a state where the F-number is opened is reproduced.

The blur amount adjustment unit 46 is similar to the blur amountadjustment unit in the third embodiment.

The display control unit 45 makes transition of display of a throughimage in the display unit 1007 from the state of FIG. 16A to the stateof FIG. 16B. In the state of FIG. 16B, an image acquired in the presentsetting of the imaging device 1400 and an image to which imageprocessing has been applied into a state in which the F-number is openedby the image processing unit 44 are displayed side by side. In FIG. 16B,The left-side image is the present image and the right-side image is theimage to which image processing has been applied into a state in whichthe F-umber is opened. Accordingly, the user can easily compare thepresent image (through image) and the image in which the degree of bluris emphasized. Note that the display form of the images is not limitedto the one illustrated in FIG. 16B, and the images may be arranged inthe vertical direction. Further, the present image and a plurality ofimages to which processing has been applied by the image processing unit44 may be displayed so that a difference occurs in the degree of blur.

Then, when any of the images is selected by the user, the displaycontrol unit 45 fully displays the selected image in the display unit1007, as illustrated in FIG. 16C. As a method of selecting an image,there are a method by selection with an operation key and a method inwhich a finger is brought into contact with any of the images in a casewhere the input unit 1008 is a touch panel.

When the selected image is fully displayed in the display unit 1007, thedisplay control unit 45 may display the blur amount adjustment functionuser interface by the blur amount adjustment unit 46 in the display unit1007. This enables the user to acquire an easily selected image by imagecapturing. Further, the blur amount adjustment function is automaticallystarted, whereby the user can adjust the blur amount such that the imagefurther suits to own taste from the state of the selected image.

Note that, in the state where a plurality of images is displayed, asillustrate in FIG. 16B, when any of the images is selected by the user,the image may be acquired by automatic image capturing. This enableseven a user who does not know the operation of the blur control tocapture a blurred image.

[4-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 4and the imaging device 1400 will be described with reference to FIG. 17.FIG. 17 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S401, an image towhich image processing has been applied into a state in which theF-number is opened is generated by the image processing unit 44.

Then, in step S402, an image acquired in the present setting of theimaging device 1400, and an image to which processing has been appliedinto a state in which the F-number is opened by the image processingunit 44 are displayed in the display unit 1007 by the display controlunit 45.

The fourth embodiment of the present technology is configured asdescribed above. According to the fourth embodiment, the user can easilycompare the present image and the image in which the degree of blur isemphasized. Accordingly, whether it is better to acquire a blurred imageby image capturing, and the like, can be easily determined.

5. Fifth Embodiment

[5-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a fifth embodiment of the present technology will be described. Inthe fifth embodiment, an image processing device 5 is also operated inan imaging device 1500. FIG. 18 is a block diagram illustrating aconfiguration of the imaging device 1500 provided with a function of theimage processing device 5 according to the fifth embodiment.

The image processing device 5 according to the fifth embodiment isdifferent from the first embodiment in that a blur related processingunit 53 functions as a composition determination unit 54, a recommendedblur amount determination unit 55, and a display control unit 56.Configurations other than the above are similar to those in the firstembodiment, and thus description is omitted.

An object to be processed of the image processing device 5 in the fifthembodiment is a frame image that configures a through image. The fifthembodiment determines a composition in an image, and applies mask(zebra) display processing to the image according to a recommended bluramount in the composition.

The composition determination unit 54 determines whether the imagematches a predetermined composition. The composition determination unit54 includes a plurality of algorithms for composition determination inadvance, and executes the composition determination processing based onthe plurality of composition determination algorithms. The compositiondetermination unit 54 determines, for example, whether an image includesa composition, such as a whole body shot, a bust shot, close-up of aface, or the like, based on the position of an object such as a person,the size of the person, the size of the face, or the like. Further, thecomposition determination unit 54 determines whether the imagecorresponds to a thirds composition, a fourths composition, a sun flagcomposition, a diagonal composition, or the like, based on apredetermined composition template. A determination result by thecomposition determination unit 54 to the recommended blur amountdetermination unit 55.

The recommended blur amount determination unit 55 determines arecommended blur amount corresponding to the composition of the imagedetermined by the composition determination unit 54. This recommendedblur amount is a recommended blur amount in the compositioncorresponding to each composition. The recommended blur amount isobtained by referring to a table in which a composition and arecommended blur amount are associated, for example. For example, asillustrated in FIG. 19A, when it has been determined by the compositiondetermination unit that an image is a thirds composition, therecommended blur amount determination unit 55 determines the recommendedblur amount corresponding to the composition.

When a difference between the recommended blur amount and the blurprediction amount is within a predetermined range, the display controlunit 56 superimposes and displays a mask on the image such that theimage becomes to have the recommended blur amount, as illustrated inFIG. 19B. This enables the user to easily recognize the state in whichan image has a recommended blur amount.

[5-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 5and the imaging device 1500 will be described with reference to FIG. 20.FIG. 20 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S501, a compositionof a present image is determined by the composition determination unit54. Next, in step S502, a recommended blur amount of the determinedcomposition is determined by the recommended blur amount determinationunit 55. Then, in step S503, when a difference between the recommendedblur amount and the blur prediction amount is within a predeterminedrange, a mask is superimposed and displayed on the image such that theimage becomes to have the recommended blur amount by the display controlunit 56.

The fifth embodiment of the present technology is configured asdescribed above. According to the fifth embodiment, a blur amount of animage can be emphasized by mask display, whereby the user can easilyrecognize what an image having a large blur amount is like.

6. Sixth Embodiment

[6-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a sixth embodiment of the present technology will be described. Inthe sixth embodiment, an image processing device 6 is also operated inan imaging device 1600. FIG. 21 is a block diagram illustrating aconfiguration of the imaging device 1600 provided with a function of theimage processing device 6 according to the sixth embodiment.

The image processing device 6 according to the sixth embodiment isdifferent from the first embodiment in that a blur related processingunit 63 functions as a recommended composition determination unit 64, agraph creation unit 65, a display control unit 66, and an imagesegmentation unit 67. Configurations other than the above are similar tothose in the first embodiment, and thus description is omitted.

An object to be processed of the image processing device 6 in the sixthembodiment is a frame image that configures a through image. The sixthembodiment presents, to the user, blur information in a recommendedcomposition in the image, with a histogram by the image processingdevice 6.

The recommended composition determination unit 64 includes a pluralityof algorithms for detecting a composition from an image, and executescomposition detection processing based on the plurality of compositiondetection algorithms.

The recommended composition determination unit 64 determines, forexample, whether an image includes a composition, such as a whole bodyshot, a bust shot, close-up of a face, or the like, based on theposition of an object such as a person, the size of the person, the sizeof the face, or the like. Further, the recommended compositiondetermination unit 64 determines whether the image includes a thirdscomposition, a fourths composition, a sun flag composition, a diagonalcomposition, or the like, based on a predetermined composition template.The recommended composition determination unit 64 then determines arecommended composition (hereinafter, referred to as recommendedcomposition) based on the position and the size of an object in eachcomposition, and the like.

The blur information acquisition unit 12 acquires a blur predictionamount of the entire image, and calculates a blur prediction amount in anon-attention area in the recommended composition determined by therecommended composition determination unit 64. The calculated blurprediction amount is supplied to the graph creation unit 65.

The graph creation unit 65 is similar to the graph creation unit of thefirst embodiment, and creates a histogram indicating the supplied blurinformation. The display control unit 66 displays the histogram createdby the graph creation unit in the display unit.

The display control unit 66 superimposes and displays the blurprediction amount on the image as a histogram in a display unit 1007, asillustrated in FIG. 22A, for example. Note that, when the blurprediction amount is displayed by the histogram, a frame indicating therecommended composition is favorably displayed together with thehistogram, as illustrated in FIG. 22A. This enables the user to easilyrecognize the position and the size of the recommended composition.

Further, when the histogram is displayed, a histogram indicating theblur prediction amount in the recommended composition and a histogram inthe entire image are favorably displayed together, as illustrated inFIG. 22B. In FIG. 22B, as an example of the display, the solid lineindicates the blur prediction amount in the entire image, and the dashedline indicates the blur prediction amount in the recommendedcomposition. The histograms are displayed in this way, the user caneasily recognize the blur prediction amount in the entire image and theblur prediction amount in the recommended composition.

The image segmentation unit 67 performs processing (trimming) ofsegmenting a range of the recommended composition determined by therecommended composition determination unit 64 from the image. With thesegmentation by the image segmentation unit, the image can have a morefavorable composition.

[6-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 6and the imaging device 1600 will be described with reference to FIG. 23.FIG. 23 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S104, object detection, determination of anattention area, and focus determination are performed after AF startprocessing.

Then, in step S105, a blur prediction amount of a non-attention area inthe entire image is acquired by a blur information acquisition unit 12.

Next, in step S601, a recommended composition is determined by therecommended composition determination unit 64. Next, in step S602, ablur prediction amount of the non-attention area in the recommendedcomposition is acquired by the blur information acquisition unit 12.Then, in step 603, a histogram indicating blur information is created bythe graph creation unit. Then, in step S604, the histogram is displayedin the display unit 1007 by the display control unit 66, as illustratedin FIGS. 22A and 22B.

Next, in step S605, a range of the recommended composition is segmentedfrom the image and a trimming image is generated by the imagesegmentation unit 67. Note that the segmentation of the image by theimage segmentation unit 67 may be automatically performed when therecommended composition is determined by the recommended compositiondetermination unit 64, or may be performed only when an instructioninput of the image segmentation is given from the user.

The sixth embodiment of the present technology is configured asdescribed above. The sixth embodiment presents the blur predictionamount in the recommended composition with a histogram while presentingthe recommended composition, whereby the user can easily recognize whatthe degree of blur of the image in the recommended composition is like.

7. Seventh Embodiment

[7-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, a seventh embodiment of the present technology will be described.In the seventh embodiment, an image processing device 7 is also operatedin an imaging device 1700. FIG. 24 is a block diagram illustrating aconfiguration of the imaging device 1700 provided with a function of theimage processing device 7 according to the seventh embodiment.

In the image processing device 7 according to the seventh embodiment, ablur related processing unit 73 functions as a recommended compositiondetermination unit 74, a recommended blur amount determination unit 75,a graph creation unit 76, a display control unit 77, and an imagesegmentation unit 78.

An object to be processed of the image processing device 7 in theseventh embodiment is a frame image that configures a through image. Theseventh embodiment presents a recommended blur amount in a recommendedcomposition and a blur prediction amount in the recommended compositionto the user with graph display. The recommended compositiondetermination unit 74 is similar to that in the sixth embodiment, anddetermines a recommended composition that is a composition ofrecommendation from an image. A determination result of the recommendedcomposition determination unit 74 is supplied to the recommended bluramount determination unit 75 and is also supplied to a blur informationacquisition unit 12. The blur information acquisition unit 12 acquires ablur prediction amount in the entire image, and also acquires a blurprediction amount in the recommended composition.

The recommended blur amount determination unit 75 determines arecommended blur amount corresponding to the recommended compositiondetermined by the recommended composition determination unit 74. Therecommended blur amount is a recommended blur amount in the composition,corresponding to each composition. The recommended blur amount is, forexample, obtained by referring to a table in which a composition and arecommended blur amount are associated.

In the seventh embodiment, first, a recommended composition in an imageis determined by the recommended composition determination unit 74.Next, the recommended blur amount determination unit 75 determines arecommended blur amount in the recommended composition. For example,when the determined recommended composition is so-called a whole bodyshot, in which the whole body of a person that is an attention areaappears, the recommended blur amount determination unit 75 determines arecommended blur amount that a blur prediction amount of a non-attentionarea is favorably small (blur prediction amount<first blur threshold).

Further, when the determined recommended composition is close-up of aface of the person that is the attention area, the recommended bluramount determination unit 75 determines a recommended blur amount thatthe blur amount of the non-attention area is favorably large (thirdthreshold<blur amount).

Further, when the determined recommended composition is so-called a bustshot of the person that is the attention area, the recommended bluramount determination unit determines a recommended blur amount that theblur amount of the non-attention area is favorably about between thewhole body shot and the close up of the face (second blur threshold<bluramount<third blur threshold). Note that a relationship among the firstto third thresholds is as described in the following expression 1:First blur threshold<Second blur threshold<Third blur threshold  (1)

The determined recommended composition and the recommended blur amountare supplied to the graph creation unit 76. The graph creation unit 76creates a graph like the one illustrated in FIGS. 25A and 25B. In theexample of the graphs of FIGS. 25A and 25B, the height of a bar graphindicates a range of the blur amount that can be realized by a presentattention area, a focal distance, and a maximum aperture of a lens.Further, a mark superimposed and displayed in a bar graph mannerindicates a blur prediction amount in the present F-number.

Further, a frame that indicates a range is displayed on the bar graph,whereby a recommended blur amount in the composition is indicated. Therecommended blur amount can present a range of the recommended bluramount by being displayed with the frame. Further, an icon presents ablur prediction amount of when an image is trimmed by the recommendedcomposition. Accordingly, the user can easily recognize what kind ofphotograph having what blur amount can be acquired by capturing thephotograph by what kind of composition. Further, the blur predictionamount in the entire image may be presented with an icon.

Further, as illustrated in FIG. 25B, when the recommended blur amountfalls outside the range of the blur amount that can be realized, tipsindicating how the blur amount is increased may be displayed by acharacter string.

The image segmentation unit 78 performs processing of segmenting therange of the recommended composition determined by the recommendedcomposition determination unit 74, from the image. With the segmentationby the image segmentation unit 78, the image can have a more favorablecomposition.

[7-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 7and the imaging device 1700 will be described with reference to FIG. 26.FIG. 26 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S701, a recommendedcomposition is determined by the recommended composition determinationunit 74. Next, in step S702, a blur prediction amount in the recommendedcomposition is acquired by the blur information acquisition unit 12.Next, in step S703, a recommended blur amount in the recommendedcomposition is determined by the recommended blur amount determinationunit 75. Next, in step S704, a graph for presenting the blur predictionamount, the recommended blur amount, and the blur prediction amount inthe recommended composition is created by the graph creation unit 76.Next, in step S705, the graph is displayed in a display unit 1007 by thedisplay control unit 77.

Then, in step S706, the range of the recommended composition issegmented from the image by the image segmentation unit 78, and atrimming image is generated. Note that the segmentation of the image bythe image segmentation unit 78 may be automatically performed when therecommended composition is determined by the recommended compositiondetermination unit 74, or may be performed only when an instructioninput of the image segmentation is given from the user.

The seventh embodiment of the present technology is configured asdescribed above. According to the seventh embodiment, a recommended bluramount in a recommended composition and a blur prediction amount in therecommended composition are presented with graph display, whereby theuser can capture an image using these pieces of information as areference.

8. Eighth Embodiment

[8-1. Configurations of an Image Processing Device and an Imaging DeviceIncluding the Image Processing Device]

Next, an eighth embodiment of the present technology will be described.In the eighth embodiment, an image processing device 8 is also operatedin an imaging device 1800. FIG. 27 is a block diagram illustrating aconfiguration of the imaging device 1800 provided with a function of theimage processing device 8 according to the eighth embodiment.

The image processing device 8 according to the eighth embodiment isdifferent from the first embodiment in that a blur related processingunit 83 functions as a segmentation range determination unit 84, arecommended blur amount determination unit 85, an image segmentationunit 86, a display control unit 87, and an enlargement processing unit88. Configurations other than the above are similar to those in thefirst embodiment, and thus description is omitted.

An object to be processed of the image processing device 8 in the eighthembodiment is a captured image acquired by an imaging device. The eighthembodiment performs segmentation processing (trimming) of the imageaccording to blur information in the image.

By trimming the captured image, the image can have a more favorablecomposition. However, a segmentation range has various shapes andpositions. Therefore, the eighth embodiment determines a segmentationrange such that the degree of blur in the image stands out according toblur information.

Segmentation of an image will be described with reference to FIGS. 28Aand 28B using a thirds composition as an example. First, thesegmentation range determination unit 84 sets a plurality of candidatesof a segmentation range by a thirds composition. For example, thecandidates of the segmentation range can be determined such that a spacehaving a predetermined size is formed in a direction where anon-attention area exists in the image.

As illustrated in FIG. 28A, a plurality of candidates exists in size andposition in a thirds composition, for example. In FIG. 28A, threesegmentation ranges are illustrated as examples of the candidates. Notethat the composition is not limited to a thirds composition. A pluralityof other composition data, such as a golden ratio composition, is storedin advance as templates, and the composition is favorably selected basedon selection of the user or other conditions. Alternatively, forexample, when a segmentation range extends outside the image in a thirdscomposition, the composition may be changed to a golden ratiocomposition.

When having determined the segmentation range candidates, thesegmentation range determination unit 84 supplies composition candidateinformation to the blur information acquisition unit 12 and therecommended blur amount determination unit 85. The blur informationacquisition unit 12 acquires a blur prediction amount of eachsegmentation range candidate, and supplies the blur prediction amount tothe segmentation range determination unit 84. Further, the recommendedblur amount determination unit 85 determines a recommended blur amountin each segmentation range candidate. The recommended blur amountdetermination unit 85 is similar to that of the seventh embodiment. Therecommended blur amount of each segmentation range candidate is suppliedto the segmentation range determination unit 84.

The segmentation range determination unit 84 determines a segmentationrange from the plurality of segmentation range candidates based on threecriteria: a composition where the blur prediction amount is closest tothe recommended blur amount; a position where the blur prediction amountin a segmentation range is closest to the recommended blur amount; and aposition where the degree of separation of the blur prediction amountsof the attention area and the non-attention area is highest. Note that aweight of each criterion may be changed for each composition.

Whether the degree of separation of the blur amounts is high can bedetermined by showing the blur prediction amounts of the attention areaand the non-attention area in each composition as histograms, asillustrated in FIG. 28C. In the example of FIG. 28C, among compositionsa, b, and c, the attention area and the non-attention area are mostseparated in the composition a. The composition a has a small trimmingarea, and thus the blur amount becomes largest when the image isenlarged. Further, an area where the blur amount is small (the portionof the face) and an area where the blur amount is large (the background)are clearly separated in the composition a, and thus the composition ahas a steep mountain like area, as illustrate d by the histogram of FIG.28C. The size of the trimming area of the composition b is almost thesame as that of the composition a. However, the larger blur area sideincludes middle to large wide ranges of blur amounts. Therefore, thecomposition b has a gently sloping mountain like area, as illustrated bythe histogram of FIG. 28C. The composition c has a large trimming area,and an emphasized blur amount is smaller than those of the compositionsa and b. A larger blur amount maintain exists in the middle of thehorizontal axis. Note that the composition a, the composition b, and thecomposition c are compositions as specific examples set for convenienceof description.

Therefore, when the composition a is selected as the segmentation range,and the segmentation of the image is executed, the image becomes the oneillustrated in FIG. 28B. By performing the segmentation processing of acaptured image in this way, the image can have a more favorablecomposition, and the degree of blur of the non-attention area can befurther emphasized.

Note that, when a diaphragm bracket function is turned ON, a diaphragmis changed suitable for each composition candidate and bracket imagecapturing may be performed. Here, a case in which one segmentation rangeis to be determined from among a plurality of composition candidatesbased on the three criteria, and the recommended blur amounts of theplurality of compositions are almost the same will be described. Imagesegmentation is performed with respect to all of the segmentation rangesof the plurality of compositions having almost the same recommended bluramounts, and trimming images are favorably presented to the user.

For example, as illustrated in FIG. 29A, when there are two compositioncandidates of a composition d and a composition e, and the recommendedblur amounts of these two compositions are almost the same, trimming isperformed for each of the compositions d and e. FIG. 29B illustrates atrimming image based on the composition d, and FIG. 29C is a trimmingimage based on the composition e.

Then, the plurality of types of trimming images is presented to theuser, and the user may be prompted to select any of the trimming images.This enables the user to acquire an image having the degree of blur thatsuits to own taste.

Note that the segmentation range of the image may be variable withrespect to an aspect of an original image. For example, when the aspectof the original image is “4:3”, the segmentation range may be changeableto “3:2” or “16:9” according to characteristics of the object. Theenlargement processing unit 88 performs processing of enlarging thetrimming image to an original image size using a super-resolutiontechnique when storing the trimming image in the storage unit 1009.Accordingly, the image has a more emphasized degree of blur than beforethe trimming.

[8-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 8and the imaging device 1800 will be described with reference to FIG. 30.FIG. 30 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S104, object detection, determination of anattention area, and focus determination are performed after AF startprocessing. Then, in step S801, a segmentation range candidate isdetermined by the segmentation range determination unit 84. Next, instep S802, a blur prediction amount of the segmentation range candidateis acquired by the blur information acquisition unit 12. Next, in stepS803, a recommended blur amount of the segmentation range candidate isdetermined by the recommended blur amount determination unit 85.

Then, in step S804, the segmentation range of the image is determined bythe segmentation range determination unit 84. Segmentation rangeinformation is supplied to the image segmentation unit 86. Next, in stepS805, the segmentation range is segmented from the image by the imagesegmentation unit 86.

Next, in step S806, the trimming image is enlarged to an original imagesize by the enlargement processing unit 88 using a super-resolutiontechnique, and is stored in the storage unit 1009. Then, a segmentedimage is displayed in a display unit 1007 by the display control unit87, as illustrated in FIG. 28B, or FIG. 29B or 29C.

Accordingly, the user can confirm the segmented image. Note that, whenthe segmented image is displayed in the display unit 1007, the image towhich the enlargement processing has been applied by the enlargementprocessing unit 88 is favorably displayed.

The eighth embodiment of the present technology is configured asdescribed above. According to the eighth embodiment, segmentation of animage is performed based on blur information in the image, whereby animage in which the degree of blur is emphasized can be created. Further,an image can be segmented in a range considering blurs in a principalobject (attention area) and a background (non-attention area), and animpressive segmented image, which is different from a composition thatincludes the entire object area, can be provided.

9. Ninth Embodiment

[9-1. Configurations of an Image Processing Device and an Imaging DeviceProvided with a Function of the Image Processing Device]

Next, a ninth embodiment of the present technology will be described. Inthe ninth embodiment, an image processing device 9 is also operated inan imaging device 1900. FIG. 31 is a block diagram illustrating aconfiguration of the image processing device 9 according to the ninthembodiment.

The image processing device 9 according to the ninth embodiment isdifferent from the first embodiment in that a blur related processingunit 93 functions as an imaging operation control unit 94 and a displaycontrol unit 95. Configurations other than the above are similar tothose in the first embodiment, and thus description is omitted. Anobject to be processed of the image processing device 9 in the ninthembodiment is a frame image that configures a through image. The imagingoperation control unit 94 controls execution of a diaphragm bracketimage capturing mode by transmitting a predetermined control signal tothe lens control unit 1002.

The diaphragm bracket image capturing mode is an image capturing mode toacquire a plurality of images having different diaphragm aperturediameters by a single image capturing operation. A plurality of imagesis acquired by changing the aperture diameters of a diaphragm, and thusa plurality of images having different blur amounts can be acquired atsingle image capturing.

When a diaphragm bracket is executed in a direction of making theF-number larger than the present setting, a plurality of images havingsmaller blur amounts in a non-attention area than a present image can beacquired, as illustrated in FIGS. 32B to 32D. In FIGS. 32A to 32D, FIG.32A illustrates an image in present setting, and the F-number becomeslarger and the images have smaller blur amounts in the non-attentionarea, in order of FIGS. 32B, 32C, and 32D.

Further, when the diaphragm bracket is executed in a direction of makingthe F-number smaller than the present setting, a plurality of imageshaving larger blur amounts in the non-attention area than the presentimage can be acquired, as illustrated in FIGS. 33B to 33D. In FIGS. 33Ato 33D, FIG. 33A illustrates an image in the present setting, and theF-number becomes smaller and the images have larger blur amounts in thenon-attention area, in order of FIGS. 33B, 33C, and 33D.

Further, the F-number is changed before and after the present setting ofthe F-number, and the diaphragm bracket can be executed. Accordingly, animage having a larger blur amount and an image having a smaller bluramount than the present image can be acquired at single image capturing.

The display control unit 95 displays the image acquired by the diaphragmbracket image capturing in a display unit 1007.

[9-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 9and the imaging device 1900 will be described with reference to FIG. 34.FIG. 34 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S901, the imagingoperation control unit 94 activates the diaphragm bracket imagecapturing mode according to the blur amount.

The imaging operation control unit 94 executes the diaphragm bracket ina direction of making the F-number larger than the present setting whenthe blur amount is a predetermined amount or more. Accordingly, aplurality of images having smaller blur amounts in the non-attentionarea than the present image can be acquired, as illustrated in FIGS. 32Bto 32D. Further, the imaging operation control unit 94 executes thediaphragm bracket in a direction of making the F-number smaller than thepresent setting when the blur amount is a predetermined amount or less.Accordingly, as illustrated in FIGS. 33B to 33D, a plurality of imageshaving larger blur amounts in the non-attention area than the presentimage can be acquired.

Next, in step S902, recording processing of the plurality of imagesacquired by the diaphragm bracket image capturing is performed by thecontrol unit 1010. The captured image is, for example, stored in astorage unit 1009 in a compressed state based on a standard such asJPEG. Note that the plurality of images captured by the diaphragmbracket image capturing may be associated and stored by tagging or thelike as mutually associated images. Further, exchangeable image fileformat (EXIF) data including information related to the stored image andadditional information such as a date and time of imaging may also bestored in association with the image. Then, in step S903, the capturedimage captured by the diaphragm bracket is displayed in the display unit1007 as a preview image by the display control unit 95. Accordingly, theuser can instantly confirm the image captured by the diaphragm bracket.

The ninth embodiment of the present technology is configured asdescribed above. According to the ninth embodiment, the diaphragmbracket image capturing is automatically executed according to the blurinformation. Therefore, for a user who does not know the existence ofthe diaphragm bracket function or a user who knows the diaphragm bracketfunction but has a low frequency of use, an image acquired by thediaphragm bracket image capturing can be provided.

10. Tenth Embodiment

[10-1. Configurations of an Image Processing Device and an ImagingDevice Provided with a Function of the Image Processing Device]

Next, a tenth embodiment of the present technology will be described. Inthe tenth embodiment, an image processing device 10 is also operated inan imaging device 2000. FIG. 35 is a block diagram illustrating aconfiguration of the imaging device 2000 provided with a function of theimage processing device 10 according to the tenth embodiment.

The image processing device 10 according to the tenth embodiment isdifferent from the first embodiment in that a blur related processingunit 103 functions as an image processing unit 104 and a display controlunit 105. Configurations other than the above are similar to those inthe first embodiment, and thus description is omitted. An object to beprocessed of the image processing device 1 in the tenth embodiment is aframe image that configures a through image. The tenth embodimentapplies shading off processing to a captured image captured by theimaging device 2000. This enables an image having a less blur amount atthe time of capturing to be an image having a large blur amount.

The image processing unit 104 applies the shading off processing to anon-attention area of the captured image based on known image shadingoff algorithm. By applying the shading off processing to thenon-attention area of the image, the image is changed from a stateillustrated in FIG. 36A to a state illustrated in FIG. 36B. In FIG. 36B,the background other than a person as an attention area is blurred,compared with FIG. 36A. When a blur amount of the non-attention area isa predetermined amount or more, the blur related processing unit 103causes the image processing unit 104 to execute background shading offprocessing.

The display control unit 105 displays an image to which the shading offprocessing has been applied by the image processing unit 104 in adisplay unit 1007.

[10-2. Processing in an Image Processing Device and an Imaging Device]

Next, a flow of processing performed in the image processing device 10and the imaging device 2000 will be described with reference to FIG. 37.FIG. 37 is a flowchart illustrating a flow of processing. Assume thatimage data to be processed has been supplied, and the focal distance andthe F-number have been acquired. Note that processing similar to that inthe first embodiment is denoted with the same step numbers anddescription is omitted.

In steps S101 to step S105, object detection, attention areadetermination, focus determination, and blur information acquisition areperformed after AF start processing. Then, in step S1001, whether a fullpress operation of a shutter has been performed by the user isdetermined. The acquisition of blur information in step S105 and thedetermination of step S1001 are repeated until the full press operationof a shutter is performed (No in step S1001).

When the full press operation of a shutter has been performed by theuser, the processing proceeds to step S1002 (Yes in step S1001). Next,in step S1002, image capturing processing is performed in the imagingdevice 2000, and an image is acquired. Next, in step S1003, whether ablur amount of when the image is captured is a predetermined value orless is determined.

When the blur amount is the predetermined value or less, the processingproceeds to step S1004 (Yes in step S1003). Then, in step S1004, shadingoff processing is applied to a non-attention area of the captured imageby the image processing unit 104. By applying the shading off processingto the image when the blur amount is the predetermined amount or less,the image processing unit 104 can make the degree of blur of thenon-attention area of the image having a less degree of blur large.Accordingly, the user can easily obtain an image having a blurbackground.

Next, in step S1005, recording processing of the image to which theshading off processing has been applied is performed. The captured imageis, for example, stored in a storage unit 1009 in a compressed statebased on a standard such as JPEG under control of a control unit 1010.Further, EXIF data including additional information, such as informationrelated to the stored image, and a date and time of imaging may also bestored in association with the image.

Then, in step S1006, the captured image to which the shading offprocessing has been applied is displayed in the display unit 1007 as apreview image by the display control unit 105. Accordingly, the user caninstantly confirm the image to which the shading off processing has beenapplied.

Description is returned to step S1003. In step S1003, when it has beendetermined that the blur amount is the predetermined about or less, theprocessing proceeds to step S1006 (No in step S1003). Then, in stepS1006, the captured image is displayed in the display unit 1007 as apreview image by the display control unit 105. Note that, in this case,steps S1004 and S1005 have not been performed, and thus the imagedisplayed in the display unit 1007 is an image to which the shading offprocessing has not been applied.

The tenth embodiment is configured as described above. According to thetenth embodiment, even the user who is not familiar with an operation ofa camera and has difficulty in capturing a blur image can easily acquirea blur image. Further, even in a case where it is difficult to acquire ablur image by diaphragm control of the imaging device, a blur image canbe acquired. According to an embodiment of the present technology,instinctive and easy-to-understand display can be provided even in anenvironment where visual observation is difficult with a liquid crystalpanel, and the like. The above-described first to tenth embodiments, theprocessing performed by the image processing devices can be executed byhardware or software. When processing by software is executed, a programin which a processing sequence is recorded is installed into a memory ofa control unit of an imaging device and executed.

For example, the program can be recorded in a recording medium, such asa hard disk or a ROM, in advance. Alternatively, the program can berecorded in a recording medium, such as a compact disc read only memory(CD-ROM), a digital versatile disc (DVD), or a semiconductor memory.Such a recording medium can be provided as packaged software. The userinstalls the packages software in the imaging device.

Note that the program that is provided in the Internet as an applicationcan be transferred and installed to the imaging device, in addition toinstallation from the above-described recording medium to the imagingdevice.

11. Modification

As described above, embodiments of the present technology have beenspecifically described. However, the present technology is not limitedto these embodiments, and various modifications based on the technicalidea of the present technology is possible.

In the above-described embodiments, description has been given taking acase of applying the present technology to an image acquired in animaging device as an example. However, the present technology is alsoapplicable to a moving image.

Further, the above first to tenth embodiments may be combined andimplemented. For example, the graph display of blur information in thefirst embodiment and the enlargement display in the second embodimentare combined, and a graph is displayed while a part of an image isenlarged and displayed.

Further, the object is not limited to a person or a human of a person,and the above-described processing may be performed using a dog, a cat,a flower, a dish, or the like, as the object, using a dog/cat facedetection function, a flower detection function, a dish detectionfunction, or the like.

Further, the present technology may employ the following configurations.

-   (1) An information processing system including: circuitry configured    to perform predetermined processing based on out-of-focus    information corresponding to a first area of an image, the    out-of-focus information corresponding to a difference in focus    between the first area of the image and a second area of the image    that is different from the first area of the image.-   (2) The system of (1), wherein the circuitry is configured to:    identify the first area of the image; and acquire the out-of-focus    information corresponding to the first area of the image.-   (3) The system of any of (1) to (2), wherein the second area of the    image is an in-focus area of the image.-   (4) The system of (1) to (3), wherein the system includes: a first    information processing apparatus; and a second information    processing apparatus configured to be connected to the first    information processing apparatus.-   (5) The system of any of (1) to (4), wherein the predetermined    processing includes outputting a notification corresponding to the    out-of focus information.-   (6) The system of any of (1) to (5), wherein the predetermined    processing includes controlling a display to display a notification    indicating an out-of-focus amount based on the out-of-focus    information.-   (7) The system of (6), wherein the circuitry is configured to    control the display to display the image and the notification    simultaneously.-   (8) The system of any of (6) to (7), wherein the circuitry is    configured to control the display to display the image with the    notification superimposed on the displayed image.-   (9) The system of any of (1) to (8), wherein the predetermined    processing includes controlling a display to display a bar graph    indicating an out-of-focus amount based on the out-of-focus    information.-   (10) The system of (9), wherein a height of the bar graph    corresponds to a range of an out-of-focus amount that may be    realized based on at least one of the second area, a focal distance    and a maximum aperture of a lens capturing the image.-   (11) The system of any of (1) to (10), wherein the predetermined    processing includes controlling a display to display an enlarged    version of the image including the first area and the second area.-   (12) The system of (11), wherein an enlargement range of the    enlarged version of the image is set so that a display area    corresponding to the first area and a display area corresponding to    the second area are substantially similar.-   (13) The system of any of (11) to (12), wherein the circuitry is    configured to control the display to display the image and the    enlarged version of the image simultaneously.-   (14) The system of any of (11) to (13), wherein the circuitry is    configured to control the display to display the enlarged version of    the image superimposed on the image.-   (15) The system of any of (1) to (14), wherein the predetermined    processing includes controlling a display to display the image while    applying a graphical effect to the first area of the image.-   (16) The system of any of (1) to (15), wherein the predetermined    processing includes trimming the image based on the out-of-focus    information.-   (17) The system of (16), wherein the circuitry is configured to    control a display to display the trimmed image.-   (18) The system of any of (1) to (17), wherein the predetermined    processing includes generating a first trimmed version of the image    and a second trimmed version of the image based on the out-of-focus    information.-   (19) The system of any of (1) to (18), wherein the predetermined    processing includes generating a first trimmed version of the image    and a second trimmed version of the image based on the out-of-focus    information.-   (20) The system of (19), wherein the circuitry is configured to    control a display to display the first trimmed version of the image    and the second trimmed version of the image.-   (21) The system of (4), wherein the first information processing    apparatus is configured to: identify the second area of the image;    acquire the out-of-focus information; and transmit the out-of-focus    information to the second information processing apparatus.-   (22) The system of (21), wherein the second information processing    apparatus is configured to: perform the predetermined processing;    and transmit a result of the predetermined processing to the first    information processing apparatus.-   (23) An information processing system including: circuitry    configured to identify a first area of an image; acquire    out-of-focus information corresponding to a second area of an image    that is different from the first area of the image; and perform    predetermined processing based on the out-of-focus information.-   (24) An information processing method including: performing    predetermined processing based on out-of-focus information    corresponding to a first area of an image, the out-of-focus    information corresponding to a difference in focus between the first    area of the image and a second area of the image that is different    from the first area of the image.-   (25) A non-transitory computer-readable medium including    computer-program instructions, which when executed by a system,    cause the system to: perform predetermined processing based on    out-of-focus information corresponding to a first area of an image,    the out-of-focus information corresponding to a difference in focus    between the first area of the image and a second area of the image    that is different from the first area of the image.-   (26) An image processing device including:-   an attention area determination unit configured to determine an    attention area that is an area of an object of attention in an image    acquired by an imaging device;-   a blur information acquisition unit configured to acquire blur    information of a non-attention area that is an area other than the    attention area in the image; and-   a blur related processing unit configured to perform predetermined    processing based on the blur information.-   (27) The image processing device according to (26), wherein the blur    related processing unit enlarges an area including the attention    area and the non-attention area and displays the enlarged area in a    display unit in the image.-   (28) The image processing device according to (26) or (27), wherein    the blur related processing unit displays a blur adjustment user    interface for adjusting the degree of blur in the image in the    display unit.-   (29) The image processing device according any of (26) to (28),    wherein the blur related processing unit applies blur processing to    the image, and displays the present image and the image to which the    blur processing has been applied in the display unit.-   (30) The image processing device according to any of (26) to (29),    wherein the blur related processing unit determines a composition of    the image, determines a recommended blur amount in the composition,    and emphasizes the degree of blur in the non-attention area of an    image to be displayed in the display unit based on the recommended    blur amount in the composition of the image and the blur information    of the non-attention area of the image and displays the image.-   (31) The image processing device according to any of (26) to (30),    wherein the blur related processing unit determines a segmentation    range of the image based on the blur information, and segments the    segmentation range from the image.-   (32) The image processing device according to (31), wherein the blur    related processing unit determines the segmentation range based on    the degree of separation of the blur information of the attention    area and the non-attention area in the segmentation range.-   (33) The image processing device according to (31) or (32), wherein,    when there is a plurality of the segmentation ranges, the blur    related processing unit segments an image with respect to each of    the plurality of segmentation ranges, and generates a plurality of    segmentation images.-   (34) The image processing device according to any of (26) to (33),    wherein the blur related processing unit performs image capturing    operation control that operates the imaging device in a bracket    image capturing mode to acquire a plurality of images having    different diaphragm aperture diameters by a single image capturing    operation, based on the blur information.-   (35) The image processing device according to (34), wherein, when    the blur amount is a predetermined amount or less, the blur related    processing unit operates the imaging device in the bracket image    capturing mode in which the diaphragm aperture diameter is changed    in a direction of becoming larger.-   (36) The image processing device according to (34), wherein, when    the blur amount is a predetermined amount or more, the blur related    processing unit operates the imaging device in the bracket image    capturing mode in which the diaphragm aperture diameter is changed    in a direction of becoming smaller.-   (37) The image processing device according to any of (26) to (36),    wherein the blur related processing unit performs, according to the    blur information of a captured image acquired by the imaging device,    image processing of emphasizing the degree of blur of the    non-attention area of the captured image.-   (38) The image processing device according to any of (26) to (37),    wherein the blur related processing unit displays a graph indicating    the blur information in the display unit.-   (39) The image processing device according to any of (26) to (38),    wherein the blur related processing unit displays a histogram    indicating the blur information in the display unit.-   (40) The image processing device according to any of (26) to (39),    wherein the blur information is an actual blur amount in the    non-attention area of the image.-   (41) The image processing device according to any of (26) to (39),    wherein the blur information is a blur prediction amount of the    non-attention area on an assumption that the imaging device has    focused the attention area.-   (42) A method of processing an image including: determining an    attention area that is an area of an object of attention in an image    acquired by an imaging device; acquiring blur information of a    non-attention area that is an area other than the attention area in    the image; and performing predetermined processing based on the blur    information.-   (43) An image processing program for causing a computer to execute a    method of processing an image, the method including: determining an    attention area that is an area of an object of attention in an image    acquired by an imaging device; acquiring blur information of a    non-attention area that is an area other than the attention area in    the image; and performing predetermined processing based on the blur    information.-   (44) An imaging device including: an image capturing unit configured    to receive light through an optical system, and to generate an    image; an attention area determination unit configured to determine    an attention area that is an area of an object of attention, in the    image; a blur information acquisition unit configured to acquire    blur information of a non-attention area that is an area other than    the attention area in the image; and-   a blur related processing unit configured to perform predetermined    processing based on the blur information. It should be understood by    those skilled in the art that various modifications, combinations,    sub-combinations and alterations may occur depending on design    requirements and other factors insofar as they are within the scope    of the appended claims or the equivalents thereof.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

REFERENCE SIGNS LIST

-   1, 2, 3, 4, 5, 6, 7, 8, 9, and 10 Image processing device-   11 Attention area determination unit-   12 Blur information acquisition unit-   13, 23, 33, 43, 53, 63, 73, 83, 93, and 103 Blur related processing    unit-   1007 Display unit-   1003 Imaging element-   1100, 1200, 1300, 1400, 1500, 1600, 1700, 1800, 1900, and-   2000 Imaging device

The invention claimed is:
 1. An information processing systemcomprising: circuitry configured to perform predetermined processingbased on out-of-focus information corresponding to a first area of animage, the out-of-focus information corresponding to a difference infocus between the first area of the image and a second area of the imagethat is different from the first area of the image, wherein thepredetermined processing includes controlling a display to display anenlarged version of the image including the first area and the secondarea.
 2. The information processing system of claim 1, wherein thecircuitry is configured to identify the first area of the image, andacquire the out-of-focus information corresponding to the first area ofthe image.
 3. The information processing system of claim 2, wherein thesecond area of the image is an in-focus area of the image.
 4. Theinformation processing system of claim 3, wherein the predeterminedprocessing includes outputting a notification corresponding to theout-of-focus information.
 5. The information processing system of claim1, wherein an enlargement range of the enlarged version of the image isset so that a first display area corresponding to the first area and asecond display area corresponding to the second area are substantiallysimilar.
 6. The information processing system of claim 1, wherein thecircuitry is configured to control the display to display the image andthe enlarged version of the image simultaneously.
 7. The informationprocessing system of claim 1, wherein the circuitry is configured tocontrol the display to display the enlarged version of the imagesuperimposed on the image.
 8. The information processing system of claim1, wherein the predetermined processing includes controlling a displayto display the image while applying a graphical effect to the first areaof the image.
 9. The information processing system of claim 1, whereinthe predetermined processing includes trimming the image based on theout-of-focus information.
 10. The information processing system of claim9, wherein the circuitry is configured to control a display to displaythe image that is trimmed.
 11. The information processing system ofclaim 1, wherein the predetermined processing includes generating afirst trimmed version of the image and a second trimmed version of theimage based on the out-of-focus information.
 12. The informationprocessing system of claim 11, wherein the circuitry is configured tocontrol a display to display the first trimmed version of the image andthe second trimmed version of the image.
 13. An information processingsystem comprising: circuitry configured to perform predeterminedprocessing based on out-of-focus information corresponding to a firstarea of an image, the out-of-focus information corresponding to adifference in focus between the first area of the image and a secondarea of the image that is different from the first area of the image,determine a composition included in the image, determine a recommendedblur amount based on the composition included in the image, determine ablur prediction amount of the first area, determine whether a differencebetween the recommended blur amount and the blur prediction amount iswithin a predetermined range, and responsive to determining that thedifference between the recommended blur amount and the blur predictionamount is not within the predetermined range, superimpose a mask on theimage to generate a through image with the recommended blur amount. 14.An information processing system comprising: circuitry configured toperform predetermined processing based on out-of-focus informationcorresponding to a first area of an image, the out-of-focus informationcorresponding to a difference in focus between the first area of theimage and a second area of the image that is different from the firstarea of the image, determine a composition included in the image,determine a recommended blur amount based on the composition included inthe image, determine a blur prediction amount of the image, generate ahistogram indicating the recommended blur amount and the blur predictionamount, and superimpose the histogram on the image to generate a throughimage with the histogram.
 15. An information processing systemcomprising: circuitry configured to perform predetermined processingbased on out-of-focus information corresponding to a first area of animage, the out-of-focus information corresponding to a difference infocus between the first area of the image and a second area of the imagethat is different from the first area of the image, determine arecommended composition for the image, determine a recommended bluramount based on the recommended composition, determine a blur predictionamount of the image, generate a graph indicating the recommended bluramount and the blur prediction amount, and superimpose the graph on theimage to generate a through image with the graph.
 16. An informationprocessing system comprising: circuitry configured to performpredetermined processing based on out-of-focus information correspondingto a first area of an image, the out-of-focus information correspondingto a difference in focus between the first area of the image and asecond area of the image that is different from the first area of theimage, determine a plurality of segmentation range candidates, determinea recommended blur amount for each of the plurality of segmentationrange candidates, determine a blur prediction amount for each of theplurality of segmentation range candidates, determine one or moresegmentation ranges from the plurality of segmentation range candidatesbased on a composition where the blur prediction amount is closest tothe recommended blur amount, a position where the blur prediction amountin the one or more segmentation ranges is closest to the recommendedblur amount, and a position where a degree of separation betweenrespective blur prediction amounts is highest, and trim the image basedon the one or more segmentation ranges to generate one or more throughimages.
 17. An information processing method comprising: performing,with circuitry, predetermined processing based on out-of-focusinformation corresponding to a first area of an image, the out-of-focusinformation corresponding to a difference in focus between the firstarea of the image and a second area of the image that is different fromthe first area of the image, wherein the predetermined processingincludes controlling a display to display an enlarged version of theimage including the first area and the second area.
 18. A non-transitorycomputer-readable medium including computer-program instructions that,when executed by a system, cause the system to perform a set ofoperations comprising: performing predetermined processing based onout-of-focus information corresponding to a first area of an image, theout-of-focus information corresponding to a difference in focus betweenthe first area of the image and a second area of the image that isdifferent from the first area of the image, wherein the predeterminedprocessing includes controlling a display to display an enlarged versionof the image including the first area and the second area.
 19. Thenon-transitory computer-readable medium of claim 18, wherein anenlargement range of the enlarged version of the image is set so that afirst display area corresponding to the first area and a second displayarea corresponding to the second area are substantially similar.
 20. Theinformation processing method of claim 17, wherein an enlargement rangeof the enlarged version of the image is set so that a first display areacorresponding to the first area and a second display area correspondingto the second area are substantially similar.